Optical imaging device for a microscope

ABSTRACT

An optical imaging device for a microscope comprises a first optical system configured to form a first optical image corresponding to a first region of a sample in accordance with a first imaging mode, a second optical system configured to form a second optical image corresponding to a second region of said sample, wherein said first and second regions spatially coincide in a target region of said sample and said first and second imaging modes are different from each other, a memory storing first distortion correction data suitable for correcting a first optical distortion caused by said first optical system in said first optical image, second distortion correction data suitable for correcting a second optical distortion caused by said second optical system in said second optical image, and transformation data suitable for correcting positional misalignment between said first and second optical images, and a processor which is configured to process first image data representing said first optical image based on said first distortion correction data for generating first distortion corrected image data, to process second image data representing said second optical image based on said second distortion correction data for generating second distortion corrected image data; and to combine said first and second distortion corrected image data based on said transformation data for generating combined image data representing a combined image which corresponds to said target region of said object.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority of European patent application number19209559.4 filed Nov. 15, 2019, the entire disclosure of which isincorporated by reference herein.

TECHNICAL FIELD

The present disclosure relates to an optical imaging device for amicroscope. Further, the disclosure relates to a method for imaging asample using a microscope and a method for calibrating an opticalimaging device for a microscope.

BACKGROUND

In the field of microscopy, a plurality of different imaging modes isused for generating optical images of a sample. Each of these imagingmodes has advantages and disadvantages in terms of image quality,spatial resolution, imaging speed, light exposure etc. For instance,whereas confocal imaging enables higher spatial resolution, it has thedisadvantageous of a relatively long image acquisition time as thesample has to be scanned point by point for creating the image. Incontrast, a wide-field imaging mode is advantageous in terms of timerequired for image acquisition. However, image resolution may besignificantly lower.

Accordingly, there is a need for combining different modes when imagingthe same sample. However, as different imaging modes may use differentoptical paths, it is a challenge to make two images generated indifferent modes coincide so that a properly combined image can bedisplayed e.g. on a monitor. In case that a user manually registers oraligns the images, similar image structures are required based on whichthe registration can be performed. This is a cumbersome task, and aprecise image fusion is not possible in many cases.

In the present context, reference is made to document EP 2 322 969 B1disclosing a microscope including a plurality of observation opticalsystems which are capable of acquiring images of the same sample indifferent modes. This microscope allows to fully utilize the individualfields of view of the plurality of observation optical systems, therebyimproving working efficiency. However, the afore-mentioned document isnot involved in automatically merging images generated by applyingdifferent imaging modes into a combined image.

SUMMARY

It is an object of the present disclosure to provide an optical imagingdevice and a method which are capable to combine images generated inaccordance with different imaging modes precisely. Further, it is anobject to provide a method for calibrating an optical imaging device fora microscope such that the optical imaging device is enabled to combineimages precisely.

The afore-mentioned objects are achieved by the subject-matter of theindependent claims. Advantageous embodiments are defined in thedependent claims and the following description.

According to an embodiment, an optical imaging device for a microscopecomprises a first optical system configured to form a first opticalimage corresponding to a first region of a sample in accordance with afirst imaging mode, a second optical system configured to form a secondoptical image corresponding to a second region of the sample, whereinthe first and second regions spatially coincide in a target region ofthe sample and the first and second imaging modes are different fromeach other. The optical imaging device further comprises a memorystoring first distortion correction data suitable for correcting a firstoptical distortion caused by the first optical system in the firstoptical image, second distortion correction data suitable for correctinga second optical distortion caused by the second optical system in thesecond optical image, and transformation data suitable for correctingpositional misalignment between the first and second optical images. Theoptical imaging device further comprises a processor which is configuredto process first image data representing the first optical image basedon the first distortion correction data for generating first distortioncorrected image data. The processor is further configured to processsecond image data representing the second optical image based on thesecond distortion correction data for generating the second distortioncorrected image data.

The processor is configured to combine the first and second distortioncorrected image data based on the transformation data for generatingcombined image data representing a combined image which corresponds tothe target region of the object.

The optical imaging device comprises two optical systems wherein each ofthese optical systems may have an image sensor which is adapted to thespecific imaging mode. It is considered that each optical system maycause an optical distortion in the optical image generated by thisoptical system. The first and second optical systems may use differentoptical paths for imaging so that the optical distortions induced by thefirst and second optical systems may be independent of each other.Accordingly, the first and second distortion correction data is notcorrelated with each other either. The distortion correction data may beindependently determined and stored for each optical system duringassembly. Typically, the optical distortion induced by the respectiveoptical system represents an aberration causing the image to be blurredor distorted so that a proper alignment of the images would be adverselyaffected. This is all the more true as the respective opticaldistortions differ significantly from each other due to the differentimaging modes. Any such adverse influence can be avoided by storing thefirst and second distortion correction data which will automatically betaken into account when merging the first and second optical images to acombined image.

The memory of the optical imaging device further stores transformationdata which is suitable for correcting positional misalignment betweenthe first and second optical images. In contrast to the first and seconddistortion correction data which can be independently determined foreach optical system, the transformation data represents data taking intoaccount both optical systems, in particular the positional relationshipbetween the optical systems possibly causing a positional misalignmentbetween the optical images.

The first and second optical systems are used to image first and secondregions of the sample, respectively, wherein these regions spatiallycoincide in a target region of the sample. The first and second regionsof the sample may spatially coincide in different ways. For instance,the first and second sample regions may be identical to each other sothat the target region itself is identical to the respective regionslikewise. Alternatively, one of the sample regions may be completelyincluded in the other region so that the target region is formed by theincluded sample region. In a further alternative, the first and secondregions may partially overlap. In such a case, the target region isformed by an overlapping area which is common to both sample regions.

The optical imaging device may comprise a display unit on which thecombined image corresponding to the imaged target region of the sampleis displayed.

Thus, the user is enabled to observe the target region of the samplebased on a synthesized image which benefits from the advantages of bothimaging modes.

Preferably, the first imaging mode is a wide-field mode, and the secondimaging mode is a confocal imaging mode. A synthesized image which isbased on these fundamentally different imaging modes provides imageinformation to the user that extends far beyond standard imageinformation.

In a preferred embodiment, the transformation data represent positionalmisalignment between a first optical reference image formed by the firstoptical system in accordance with the first imaging mode and a secondoptical reference image formed by the second optical system inaccordance with the second imaging mode. The optical reference imagesmay be generated using a reference object which is adapted to beproperly imaged in both imaging modes. Alternatively, live imagesgenerated during the actual imaging process may be used as referenceimages.

Preferably, a calibration mode is provided in which the processor isconfigured to generate the transformation data and to store thetransformation data in the memory prior to forming the first and secondoptical images. In this calibration mode, the processor may further beconfigured to generate the first and second distortion correction dataand to store this data in the memory. Preferably, the calibration modeis applied in the manufacturing and assembling process so that thedistortion correction data and the transformation data is already storedin the finished product. Accordingly, the user does not need to worryabout any calibration. Rather, the user is allowed to fully concentrateon the experiment including sample preparation, adjusting imagingparameters etc.

The processor may be configured to determine correlation datarepresenting a correlation between the first and second opticalreference images and to generate the transformation data based on saidcorrelation data. For instance, an algorithm may be applied whichdetermines a correlation coefficient based on an identification ofstructural features in the images. Based on this information, aniterative optimization procedure may be used to determine a requiredcoordinate transformation.

The positional misalignment represented by the transformation data maycomprise translation, rotation, scaling, shearing, mirroring, and/ordistortion.

In a preferred embodiment, the processor is configured to update thetransformation data and to store the updated transformation data in thememory. By updating the transformation data, the user is enabled toreact to changes which occur during the experiment. For instance,changes due to drift, structural modifications, dynamic processes in thesample, etc. can be compensated by redetermining the transformationstored in the memory.

As an example, the processor may be configured to cause the firstoptical system to generate a sequence of first optical images inaccordance with the first imaging mode and to cause the second opticalsystem to generate a sequence of second optical images in accordancewith the second imaging mode. For redetermining the transformation, theprocessor may further be configured to determine a first tracking markwithin one of the first optical images and to determine a secondtracking mark within one of the second optical images. In such a case,the processor is configured to perform tracking of the first trackingmark and the second tracking mark and to update the transformation databased on said tracking. By automatically defining so-called landmarks inform of the afore-mentioned tracking marks in initial images and bytracking these landmarks over time, it is possible to recalibrate thetransformation without having to use a calibration standard in form ofthe reference object again.

The processor may further be configured to combine the first and seconddistortion corrected image data based on the transformation data suchthat one of the first and second optical images is mapped to a referencesystem defined by the other of said first and second optical images orsuch that both optical images are mapped to a common reference system.In other words, a coordinate transformation is applied by which thecoordinate system of one optical image is transformed into thecoordinate system of the other image or the coordinate systems of bothoptical images are transformed into a new common coordinate system.

The first and second optical systems may be installed in a fixedpositional relationship to each other. Preferably, the two differentimaging modes are adapted as closely as possible to each other byphysically aligning the respective optical elements. Such an alignmentallows to minimize the required transformation of the image points.Hence, interferences due to interpolation, rotation etc. may besignificantly reduced.

According to another aspect, a method for imaging a sample using amicroscope is provided. The imaging method comprises the followingsteps: forming a first optical image corresponding to a first region ofa sample in accordance with a first imaging mode by means of a firstoptical system; forming a second optical image corresponding to a secondregion of the sample by means of a second optical system, wherein thefirst and second regions spatially coincide in a target region of thesample and the first and second imaging modes are different from eachother; obtaining first distortion correction data suitable forcorrecting a first optical distortion caused by the first optical systemin the first optical image; obtaining second distortion correction datasuitable for correcting a second optical distortion caused by the secondoptical system in the second optical image; obtaining transformationdata suitable for correcting positional misalignment between the firstand second optical images; processing first image data representing thefirst optical image based on the first distortion correction data forgenerating first distortion corrected image data; processing secondimage data representing the second optical image based on the seconddistortion correction data for generating a second distortion correctedimage data; and combining the first and second distortion correctedimage data based on the transformation data for generating combinedimage data representing a combined image which corresponds to the targetregion of the object.

According to another aspect, a method for calibrating an optical imagingdevice for a microscope is provided, said optical imaging devicecomprising a first optical system configured to form a first opticalimage corresponding to a first region of a sample in accordance with afirst imaging mode; a second optical system configured to form a secondoptical image corresponding to a second region of the sample, whereinthe first and second regions spatially coincide in a target region ofthe sample and the first and second imaging modes are different fromeach other; a processor; and a memory. The method includes the followingcalibration steps: obtaining first distortion correction data suitablefor correcting a first optical distortion caused by the first opticalsystem in the first optical image; obtaining second distortioncorrection data suitable for correcting a second optical distortioncaused by the second optical system in the optical image; obtainingtransformation data suitable for correcting positional misalignmentbetween the first and second optical images; and storing the firstdistortion correction data, the second distortion correction data andthe transformation data in the memory to be accessible by the processor.

The calibration method may be performed in the course of themanufacturing and assembling process so that the assembled product canbe provided to the user with the required calibration data alreadystored therein.

According to a preferred embodiment, a first optical reference image ofa reference object is formed by means of the first optical system inaccordance with the first imaging mode. A second optical reference imageof the reference object is formed by means of the second optical systemin accordance with the second imaging mode. The transformation data isdetermined based on positional misalignment of the first and secondreference images. By using a single calibration standard in form of theafore-mentioned reference object, the transformation data can bedetermined in a simple and reproducible manner. The calibration standardcan also be used for determining the first and second distortioncorrection data.

Just as an example, the reference object may comprise a grid formed by aplurality of spots. The grid is adapted to the first and second opticalsystems such that each optical system is capable to image at least twoof the plurality of spots over a range of available magnifications. Sucha grid is used to ensure that the calibration standard containssufficient structure information to achieve the desired degree ofaccuracy. In particular, the grid is formed to represent identicalstructures in both imaging modes. Further, the grid may be imaged withboth transmitted light and fluorescence light.

BRIEF DESCRIPTION OF THE DRAWING VIEWS

Hereinafter, specific embodiments are described referring to thedrawings, wherein:

FIG. 1 is a schematic diagram showing an optical imaging device for amicroscope according to an embodiment;

FIG. 2 is a diagram illustrating examples for a positional relationshipbetween first and second regions of a sample to be imaged by the opticalimaging device;

FIG. 3 is a diagram showing a reference object used for calibrationaccording to an embodiment;

FIG. 4 is a diagram illustrating a tracking process for updatingtransformation data;

FIG. 5 is a flow diagram showing a method for calibrating the opticalimaging device according to an embodiment;

FIG. 6 is a flow diagram showing a method for imaging a sample using themicroscope according to an embodiment; and

FIG. 7 is a diagram illustrating a transformation of coordinate systemsassigned to first and second optical images, respectively, into a commonreference coordinate system.

DETAILED DESCRIPTION

FIG. 1 shows a microscope 100 comprising an optical imaging device 102which is configured to image a sample 104 on a microscope stage 106 inaccordance with different imaging modes. For instance, the opticalimaging device 102 may serve to image the sample 104 in a wide-fieldimaging mode and a confocal imaging mode. Needless to say that theseimaging modes are to be understood only as examples. Any other modes maybe applied as long as these modes are adapted to provide imageinformation which can be sensibly combined for generating a synthesizedimage.

The optical imaging device 102 comprises a first optical system 108which is configured to form a first optical image corresponding to afirst region 210 (see

FIG. 2) of the sample 104 in accordance with a first imaging mode beingthe wide-field imaging mode in the present embodiment. Likewise, theoptical imaging device 102 comprises a second optical system 112 whichis configured to form a second optical image corresponding to a secondregion 214 (see FIG. 2) of the sample 104 in accordance with a secondimaging mode being the confocal imaging mode in the present embodiment.As schematically illustrated in FIG. 1, the first and second opticalsystems 108, 112 use different optical paths 116 and 118, respectively,along which detection light emanating from the sample 104 propagatesthrough the optical system 108, 112 towards image sensors 120, 122 whichare coupled to the optical systems 108, 112, respectively. The firstimage sensor 120 which is assigned to the first optical system 108 maybe formed by a camera suitable for wide-field imaging. The second imagesensor 122 which is assigned to the second optical system 112 may beformed by a sensor suitable for confocal imaging, e.g. a point detector.

The optical imaging device 102 further comprises a processor 124 whichmay serve to control the overall operation of the optical imaging device102. In particular, the processor 124 is configured to process firstimage data representing a first optical image and second image datarepresenting a second optical image, wherein the first and secondoptical images are generated on the image sensors 120, 122 by means ofthe first and second optical systems 108, 112, respectively. For this,the processor 124 is connected to the optical systems 108, 112 and theimage sensors 120, 122 via control lines 126, 128, 130, 132.

The optical imaging device further comprises a memory 134 connected tothe processor 124 via a control line 136. Further, a display unit 138may be provided which is connected to the processor 124 via a controlline 140.

Furthermore, in order to provide the different optical paths 116, 118towards the first and second optical systems 108, 112, a beam splitteror any other suitable light deflector 142 may be included in the opticalimaging device 102.

As mentioned above, the optical imaging device 102 is able to operate inthe wide-field imaging mode and the confocal imaging mode in order toimage the first region 210 and the second region 214 of the sample 104.The first and second regions 210, 214 spatially coincide in a targetregion of the sample 104 which is illustrated by a hatched area 242 inFIG. 2. The spatial coincidence may be realized in different ways. Forexample, the second region 214 assigned to the confocal imaging mode maybe completely included in the first region as shown in FIG. 2(a). As thetarget region 242 is formed by an overlap of the first and secondregions 210, 214, the target region 242 is identical to the secondregion 214 in the example illustrated in FIG. 2(a). Further, FIG. 2(b)shows an example in which the first and second regions 210, 214 areidentical so that the target region 242 is identical to the respectiveregions likewise. In the example shown in FIG. 2(c), the first andsecond regions 210, 214 partially overlap so that the target region 242is formed by an overlapping area which is common to both regions 210,214. Needless to say that the spatial coincidence between the first andsecond regions 210, 214 is not limited to the examples shown in FIG. 2.

In the embodiment shown in FIG. 1, the memory 134 is provided forstoring first distortion correction data and second distortioncorrection data. The first distortion correction data is suitable forcorrecting a first optical distortion, e.g. an optical aberration, whichis caused by the first optical system 108 when generating the firstoptical image of the first region 210 of the sample 104 in accordancewith the wide-field imaging mode. Likewise, the second distortioncorrection data serves to correct a second optical distortion which iscaused by the second optical system 112 when generating the secondoptical image of the second region 214 of the sample 104 in accordancewith the confocal imaging mode. As the first and second optical systems108, 112 use the different optical paths 116, 118 for imaging therespective region 210, 214, the optical distortions induced by the firstand second optical systems 108, 112 are independent of each other.Hence, the first and second distortion correction data can beindependently determined and stored in the memory 134.

Further, the memory 134 stores transformation data which can be used forcorrecting positional misalignment between the first and second opticalimages created in the wide-field imaging mode and the confocal imagingmode, respectively. Whereas the first and second distortion correctiondata can be independently assigned to each optical system 108, 112, thetransformation data stored in the memory 134 reflects the positionalrelationship between the optical systems 108, 112.

The processor 124 utilizes the first and second distortion correctiondata as well as the transformation data stored in the memory 134 forproviding a combined image which corresponds to the target region 242 ofthe object 104. This combined image provides image information derivedfrom both wide-field imaging and the confocal imaging. In order tocreate the combined image, the processor generates first distortioncorrected image data by processing first image data representing thefirst optical image based on the first distortion correction data.Likewise, the processor 124 generates second distortion corrected imagedata by processing second image data representing the second opticalimage based on the second distortion correction data. Then, based on thetransformation data, the processor combines the first and seconddistortion correction image data in order to create combined image datawhich represent the combined image to be displayed on the display unit134.

The optical imaging device 102 may provide a calibration mode in whichthe processor 124 generates the transformation data and stores this datain the memory 134. This calibration mode is preferably applied in theprocess of manufacturing and assembling the optical imaging device 102so that it can be automatically used at a later stage when a useroperates the microscope 100 for imaging the sample 104. The first andsecond distortion correction data independently assigned to therespective optical systems 108, 112 may be generated by the processor124 in the calibration mode likewise.

For calibrating the optical imaging device 102 a reference object may beused. Just as an example, such a reference object may be formed by agrid 350 as illustrated in FIG. 3.

The grid 350 comprises a plurality of spots 352 which are provided in arectangular array. The grid 350 is adapted to the first and secondoptical systems 108, 112 such that both optical systems 108, 112 arecapable to image at least two of the spots 352 over an availablemagnification range of the microscope 100 despite the fact that theoptical systems 108, 122 apply different imaging modes.

For the purpose of calibration, the first optical system 108 generates afirst optical reference image of the grid 350 in the wide-field imagingmode. Correspondingly, the second optical system 112 generates a secondoptical reference image of the grid 350 in the confocal imaging mode.Subsequently, the processor 124 generates the first and seconddistortion correction data and stores the data in the memory 134.Furthermore, the processor determines a positional misalignment betweenthe first and second reference images representing the grid 350. Basedon this misalignment, the processor 124 generates the transformationdata and stores the data in the memory 134.

Using a reference object as shown in FIG. 3 for calibrating the opticalimaging device 102 is to be understood merely as an example. Thus, thetransformation data may also be generated based on live images which arecreated when the user operates the microscope 100 for imaging the sample104. Further, live images may also be used in order to update theinitial transformation data stored in the memory 134 in order tocompensate for drift, structural modification, dynamic processes in thesample, etc. occurring during an experiment. For instance, the processor124 may cause each of the optical systems 108, 112 to generate asequence of optical images in accordance with the respective imagingmode. For each sequence, the processor determines a tracking mark 452 asillustrated in FIG. 4 within an initial image of this sequence andtracks the tracking mark 452 over time, i.e. over a plurality of imageswhich are generated subsequent the initial image. Using the trackingmark 452, the processor 124 is enabled to generate updatedtransformation data for recalibrating the transformation without havingto use a reference object as shown in FIG. 3.

In order to generate the transformation data, the processor 124 may beconfigured to determine correlation data which represents a correlationbetween the reference images. As explained above, images representing areference object or live images may be used.

The flow diagram of FIG. 5 illustrates a method for calibrating theoptical imaging device 102 according to an embodiment.

In step S2, the first optical system 108 images a reference object asillustrated in FIG. 3 for forming a first reference image. Image datarepresenting the first reference image may be stored in an image memory(not shown in the Figures). In step S4, the processor 124 determines thefirst distortion correction data e.g. by comparing digital datarepresenting the first reference image with nominal data which have beendefined in advance to correspond to an ideal reference image, i.e. to animage which would be generated without any optical distortion. In stepS6, the processor 124 stores the first distortion correction data in thememory 134.

In step S8, the second optical system 112 forms a second referenceimage, and the processor 124 stores corresponding image data in theimage memory. In step S10, the processor 124 determines the seconddistortion correction data. In step S12, the processor 124 stores thesecond distortion correction data in the memory 134. The steps S8, S10,and S12 for generating and storing the second distortion correction dataare performed in the same manner as steps S2, S4, and S6 with respect tothe first distortion correction data.

In step S14, the processor 124 determines the transformation data basedon the first and second reference images as explained above. Finally, instep S16, the processor 124 stores the transformation data in the memory134.

The flow diagram shown in FIG. 6 illustrates a method for imaging thesample 104 according to an embodiment.

In step S12, the first optical system 108 forms the first optical imageof the first target region 210 in accordance with the wide-field imagingmode. Correspondingly, in step 14, the second optical system 112 formsthe second optical image of the second region 214 in accordance with theconfocal imaging mode. As explained above with reference to FIG. 2, thefirst and second regions of the sample 104 spatially coincide in thetarget region 242. First and second image data representing the firstand second optical images, respectively, are stored in the image memoryin Steps 12 and 14.

After the first and second images have been generated, the processor 124reads the first distortion correction data from the memory 134 in stepS16. Likewise, in step S18, the processor 124 reads the seconddistortion correction data from the memory 134. In step S20, theprocessor 124 reads the transformation data from the memory 134.

In step S22, the processor 124 processes the first image datarepresenting the first optical image based on the first distortioncorrection data in order to create first distortion corrected imagedata. Correspondingly, in step 24, the processor 124 processes thesecond image data representing the second optical image based on thesecond distortion correction data in order to create second distortioncorrected image data. The first and second distortion corrected imagedata are stored in the image memory.

Finally, in step S26, the processor 124 combines the first and seconddistortion corrected image data using the transformation data read fromthe memory 134. Thus, combined image data are created which represent acombined image. The combined image corresponding to the target region242 of the sample 104 may be displayed on the display unit 138. Further,the combined data may be stored in the image memory not shown in theFigures.

A specific transformation performed by the processor 124 for combiningthe first and second optical images is schematically illustrated inFIGS. 7. According to the example of FIG. 7, the processor 124 uses twomapping operation T′ and T″ for transforming a first coordinate systemKS1 assigned to the first image and a second coordinate system KS2assigned to the second image into a third coordinate system KS3. Thethird coordinate system KS3 represents a common reference system whichis assigned to the combined image formed from the first and secondimages.

Needless to say the transformation shown in FIG. 7 is merely an example.Thus, the processor 124 may combine the first and second distortioncorrected image data based on the transformation data such that thefirst coordinate system KS1 is mapped into the second coordinate systemKS2 or vice versa.

Although some aspects have been described in the context of anapparatus, it is clear that these aspects also represent a descriptionof the corresponding method, where a block or device corresponds to amethod step or a feature of a method step. Analogously, aspectsdescribed in the context of a method step also represent a descriptionof a corresponding block or item or feature of a correspondingapparatus. Some or all of the method steps may be executed by (or using)a hardware apparatus, like for example, a processor, a microprocessor, aprogrammable computer or an electronic circuit. In some embodiments,some one or more of the most important method steps may be executed bysuch an apparatus.

Depending on certain implementation requirements, embodiments of thedisclosure can be implemented in hardware or in software. Theimplementation can be performed using a non-transitory storage mediumsuch as a digital storage medium, for example a floppy disc, a DVD, aBlu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory,having electronically readable control signals stored thereon, whichcooperate (or are capable of cooperating) with a programmable computersystem such that the respective method is performed. Therefore, thedigital storage medium may be computer readable.

Some embodiments according to the disclosure comprise a data carrierhaving electronically readable control signals, which are capable ofcooperating with a programmable computer system, such that one of themethods described herein is performed.

Generally, embodiments of the present disclosure can be implemented as acomputer program product with a program code, the program code beingoperative for performing one of the methods when the computer programproduct runs on a computer. The program code may, for example, be storedon a machine readable carrier.

Other embodiments comprise the computer program for performing one ofthe methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the present disclosure is, therefore, acomputer program having a program code for performing one of the methodsdescribed herein, when the computer program runs on a computer.

A further embodiment of the present disclosure is, therefore, a storagemedium (or a data carrier, or a computer-readable medium) comprising,stored thereon, the computer program for performing one of the methodsdescribed herein when it is performed by a processor. The data carrier,the digital storage medium or the recorded medium are typically tangibleand/or non-transitionary. A further embodiment of the present disclosureis an apparatus as described herein comprising a processor and thestorage medium.

A further embodiment of the disclosure is, therefore, a data stream or asequence of signals representing the computer program for performing oneof the methods described herein. The data stream or the sequence ofsignals may, for example, be configured to be transferred via a datacommunication connection, for example, via the internet.

A further embodiment comprises a processing means, for example, acomputer or a programmable logic device, configured to, or adapted to,perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon thecomputer program for performing one of the methods described herein.

A further embodiment according to the disclosure comprises an apparatusor a system configured to transfer (for example, electronically oroptically) a computer program for performing one of the methodsdescribed herein to a receiver. The receiver may, for example, be acomputer, a mobile device, a memory device or the like. The apparatus orsystem may, for example, comprise a file server for transferring thecomputer program to the receiver.

In some embodiments, a programmable logic device (for example, a fieldprogrammable gate array) may be used to perform some or all of thefunctionalities of the methods described herein. In some embodiments, afield programmable gate array may cooperate with a microprocessor inorder to perform one of the methods described herein. Generally, themethods are preferably performed by any hardware apparatus.

LIST OF REFERENCE SIGNS

100 microscope

102 optical imaging device

104 sample

106 microscope stage

108 first optical system

112 second optical system

116 optical path

118 optical path

120 image sensor

122 image sensor

124 processor

126 control line

128 control line

130 control line

132 control line

134 memory

136 control line

138 display unit

140 control line

142 light deflector

210 first region

214 second region

242 target region

350 grid

352 spots

452 tracking mark

KS1 first coordinate system

KS2 second coordinate system

KS3 third coordinate system

T transformation

T′ transformation

What is claimed is:
 1. An optical imaging device (102) for a microscope(100), comprising: a first optical system (108) configured to form afirst optical image corresponding to a first region (210) of a sample(104) in accordance with a first imaging mode; a second optical system(112) configured to form a second optical image corresponding to asecond region (214) of said sample (104), wherein said first and secondregions (210, 214) spatially coincide in a target region (242) of saidsample (104) and said first and second imaging modes are different fromeach other; a memory (134) storing first distortion correction datasuitable for correcting a first optical distortion caused by said firstoptical system (108) in said first optical image, second distortioncorrection data suitable for correcting a second optical distortioncaused by said second optical system (112) in said second optical image,and transformation data suitable for correcting positional misalignmentbetween said first and second optical images, and a processor (124)which is configured to: process first image data representing said firstoptical image based on said first distortion correction data forgenerating first distortion corrected image data, process second imagedata representing said second optical image based on said seconddistortion correction data for generating second distortion correctedimage data; and combine said first and second distortion corrected imagedata based on said transformation data for generating combined imagedata representing a combined image which corresponds to said targetregion (242) of said sample (104).
 2. The optical imaging device (102)according to claim 1, wherein said first imaging mode is wide-fieldimaging mode and said second imaging mode is a confocal imaging mode. 3.The optical imaging device (102) according to claim 1, wherein saidtransformation data represents positional misalignment between a firstoptical reference image formed by said first optical system (108) inaccordance with said first imaging mode and a second optical referenceimage formed by said second optical system (112) in accordance with saidsecond imaging mode.
 4. The optical imaging device (102) according toclaim 1, wherein a calibration mode is provided in which said processor(124) is configured to generate said transformation data and to storesaid transformation data in said memory (134) prior to forming saidfirst and second optical images.
 5. The optical imaging device (102)according to claim 3, wherein said processor (124) is configured todetermine correlation data representing a correlation between said firstand second optical reference images and to generate said transformationdata based on said correlation data.
 6. The optical imaging device (102)according to claim 1, wherein the positional misalignment represented bythe transformation data comprises translation, rotation, scaling,shearing, mirroring, and/or distortion.
 7. The optical imaging device(102) according to claim 1, wherein said processor (124) is configuredto update said transformation data and to store said updatedtransformation data in said memory (134).
 8. The optical imaging device(102) according to claim 7, wherein said processor (124) is configuredto cause said first optical system (108) to generate a sequence of firstoptical images in accordance with said first imaging mode and to causesaid second optical system (112) to generate a sequence of secondoptical images in accordance with said second imaging mode, wherein saidprocessor (124) is configured to determine a first tracking mark (452)within one of said first optical images and to determine a secondtracking mark within one of said second optical images, and wherein saidprocessor (124) is configured to perform tracking of said first trackingmark and said second tracking mark (452) and to update saidtransformation data based on said tracking.
 9. The optical imagingdevice (102) according to claim 1, wherein said processor (124) isconfigured to combine said first and second distortion corrected imagedata based on said transformation data such that one of said first andsecond optical images is mapped to a reference system (KS1, KS2) definedby the other of said first and second optical images or such that bothoptical images are mapped to a common reference system (K53).
 10. Theoptical imaging device (102) according to claim 1, wherein said firstand second optical systems (108, 112) are installed in a fixedpositional relationship to each other.
 11. A method for imaging a sample(104) using a microscope (100), comprising the following steps: forminga first optical image corresponding to a first region (210) of a sample(104) in accordance with a first imaging mode using a first opticalsystem (108); forming a second optical image corresponding to a secondregion (214) of said sample (104) in accordance with a second imagingmode using a second optical system (112), wherein said first and secondregions (210, 214) spatially coincide in a target region (242) of saidsample (104) and said first and second imaging modes are different fromeach other; obtaining first distortion correction data suitable forcorrecting a first optical distortion caused by said first opticalsystem (108) in said first optical image; obtaining second distortioncorrection data suitable for correcting a second optical distortioncaused by said second optical system (112) in said second optical image;obtaining transformation data suitable for correcting positionalmisalignment between said first and second optical images; processingfirst image data representing said first optical image based on saidfirst distortion correction data for generating first distortioncorrected image data; processing second image data representing saidsecond optical image based on said second distortion correction data forgenerating second distortion corrected image data; and combining saidfirst and second distortion corrected image data based on saidtransformation data for generating combined image data representing acombined image which corresponds to said target region (242) of saidsample (104).
 12. A method for calibrating an optical imaging device(102) for a microscope (100), said optical imaging device (102)comprising: a first optical system (108) configured to form a firstoptical image corresponding to a first region (210) of a sample (104) inaccordance with a first imaging mode, a second optical system (112)configured to form a second optical image corresponding to a secondregion (214) of said sample (104), wherein said first and second regions(214) spatially coincide in a target region (242) of said sample (104)and said first and second imaging modes are different from each other, aprocessor (124), and a memory (134), wherein said method includesfollowing calibration steps: obtaining first distortion correction datasuitable for correcting a first optical distortion caused by said firstoptical system (108) in said first optical image; obtaining seconddistortion correction data suitable for correcting a second opticaldistortion caused by said second optical system (112) in said secondoptical image; obtaining transformation data suitable for correctingpositional misalignment between said first and second optical images;and storing said first distortion correction data, said seconddistortion correction data and said transformation data in said memory(134) to be accessible by said processor (124).
 13. The method accordingto claim 12, wherein a first optical reference image of a referenceobject (350) is formed using said first optical system (108) inaccordance with said first imaging mode, a second optical referenceimage of said reference object is formed using said second opticalsystem (112) in accordance with said second imaging mode, and saidtransformation data is determined based on positional misalignment ofsaid first and second optical reference images.
 14. The method accordingto claim 13, wherein said reference object comprises a grid (350) formedby a plurality of spots (352), said grid (350) being adapted to saidfirst and second optical systems (112) such that each of the first andsecond optical systems (108, 112) is capable of imaging at least two ofsaid plurality of spots (352) over a range of available magnifications.15. A non-transitory computer-readable medium storing a computer programcomprising instructions which, when the instructions are executed by aprocessor, cause the processor to perform the method according to claim11.